Structural Estimation of Partially Observable Markov Decision Processes
نویسندگان
چکیده
Partially Observable Markov Decision Processes (POMDPs) is a well-developed framework for sequential decision making under uncertainty and partial information. This paper considers the (inverse) structural estimation of primitives POMDP based upon data in form sequences observables implemented actions. We analyze properties an entropy regularized specify conditions which model identifiable without knowledge state dynamics. consider soft policy gradient algorithm to compute maximum likelihood estimator, illustrate approach with equipment replacement problem.
منابع مشابه
Partially observable Markov decision processes
For reinforcement learning in environments in which an agent has access to a reliable state signal, methods based on the Markov decision process (MDP) have had many successes. In many problem domains, however, an agent suffers from limited sensing capabilities that preclude it from recovering a Markovian state signal from its perceptions. Extending the MDP framework, partially observable Markov...
متن کاملAmbiguous Partially Observable Markov Decision Processes: Structural Results and Applications
Markov Decision Processes (MDPs) and their generalization, Partially Observable MDPs (POMDPs), have been widely studied and used as invaluable tools in dynamic stochastic decision-making. However, two major barriers have limited their application for problems arising in various practical settings: (a) computational challenges for problems with large state or action spaces, and (b) ambiguity in ...
متن کاملBounded-Parameter Partially Observable Markov Decision Processes
The POMDP is considered as a powerful model for planning under uncertainty. However, it is usually impractical to employ a POMDP with exact parameters to model precisely the real-life situations, due to various reasons such as limited data for learning the model, etc. In this paper, assuming that the parameters of POMDPs are imprecise but bounded, we formulate the framework of bounded-parameter...
متن کاملQuantum partially observable Markov decision processes
Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. We present quantum observable Markov decision processes (QOMDPs), the quantum analogs of partially observable Marko...
متن کاملInducing Partially Observable Markov Decision Processes
In the field of reinforcement learning (Sutton and Barto, 1998; Kaelbling et al., 1996), agents interact with an environment to learn how to act to maximize reward. Two different kinds of environment models dominate the literature—Markov Decision Processes (Puterman, 1994; Littman et al., 1995), or MDPs, and POMDPs, their Partially Observable counterpart (White, 1991; Kaelbling et al., 1998). B...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Automatic Control
سال: 2022
ISSN: ['0018-9286', '1558-2523', '2334-3303']
DOI: https://doi.org/10.1109/tac.2022.3217908